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Evidence-Based Language Practice 



The year is 2025. The student asks his teacher, Mrs. Mortensen. “How iong wiii It take me to 
write correct journai entries?” or “Wiii I become abie to speak at meetings if I practice sociai 
speaking?” or “Shouid I spend a iot of time on the past imperfect tense if I want to speak to 
other peopie?” The teacher then teiis the student she must get some data to answer the 
questions but wiii provide an answer for the next ciass. The teacher then goes to the Engiish 
Department’s computer and forms a iiterature search. For one question, there is an exciting 
iiterature review that is in one of the on-iine ianguage journais. The review is recent (within the 
iast 2 months) and her search of the iiterature within the time reveais no new reievant studies. 
For the other questions, the department’s on-iine manuscript service provides her with the 
ianguage studies she seiects. The search and downioad take iess than 30 minutes. Review of 
the iiterature aiiows her to answer the student’s questions. She then fiies the computer search 
on the department’s evidence-based iibrary so others may access it when the need arises. She 
makes a note in the eiectronic database as to the avaiiabiiity of the new review and another 
annotation of where the data are stored. Is this a far-fetched scenario? I think not. 

Medicai education of doctors and patients has iong taught and used research as the basis for 



deveiopment of protocois, criticai pathways, care maps, and standards of care (Strauss 1988). 




I believe that language education of teachers and students can also use research as the basis 



for the establishment of educational guidelines, curricula, syllabi, standards, lesson plans for the 
classroom, and classroom activities. Retrieving research literature and critically analyzing it 
have been an integral part of planning for classes or populations of students in the medical 
profession (Oxman, Sackett, and Guyatt 1993). It usually involves reading the literature and 
selecting the portions to apply to students. Often subjective criteria were not revealed in the final 
distribution or conclusion from the review. Rarely did the teacher in the classroom, nor the 
research professor answer questions in the classroom based on literature reviewed specific to 
an individual’s situation. 

Evidence-based language practice has not been introduced as of yet. It focuses on providing 
answers to student questions and answers from others based on the most recent and valid 
literature. The terminology used is usually from the field of linguistics and thus is often not 
familiar to the teacher who has been prepared with a liberal arts or social science perspective 
on education. It is important to realize why the concept of evidence-based education should be 
developed and promulgated. The rapid explosion in knowledge and recent changes in 
communication and technology have worked synergistically to make this opportunity available 
and viable. Computer systems have been built for storage and access to new literature. The 



internet has made knowledge readily available. It has created a whole new communication 




system. With the availability of these new tools, there has been a realization that research 



data can and should be the basis of practical classroom management. 

Historically, teachers have answered questions and planned instruction by first reflecting on 
their own practices, consulting a text, looking at the relevant portion of the institution’s 
procedure book, or perhaps even asking an expert (Black 1998). The standard of instruction 
was based on an intuitive background, a strong grounding in educational methodology and 
years of teaching practice. When teachers wanted an answer, they looked up the question in a 
text or consulted an expert. What was leaned in school is fine but currently (days, weeks, or 
years later), the information may be inaccurate, incomplete, outdated, or outmoded. Current 
research findings exist and are readily available to answer questions from students, teachers 
and others. We simply need to access them to determine whether they meet our specific 
educational needs. 

Language education should embrace evidence-based practice but not discard the fine 
preparation in research that has taken place in schools of education over the years. The 
understanding of research design and critical analysis of studies that has been an integral pat of 
educational traditions and must be retained as we embrace the principles of evidence-based 



practice. If we are able to understand evidence-based principles and integrate them with our 




current critical analysis of studies, evidence-based educational practice holds the promise of 



making radical positive changes in education. We must keep what has worked and build on it. 



The Methodology 

Evidence-based practice is an approach to teaching and learning that is based on knowledge 
and strength of the evidence on which practice is based (McGuckin 1998). It involves defining a 
problem, identifying the needed information, conducting a literature search, critically appraising 
the literature using the principles of evidence based practice, identifying the applicable data, and 
applying it to the student (see Box 1). 

Evidence-based practice demands that the teacher use the educational acumen developed in 
school and classroom experience, including current knowledge of such areas as psychology 
and sociology to determine whether the findings of the research are applicable to the student or 
student population at large. This strategy does not replace classroom expertise; it enhances it. 



Box 1. Steps in the evidence-based approach 

1 ) Define the problem 6) Determine if it applies to the situation 

2) Identify-the needed information 7) Apply it to the student, when appropriate 

3) Conduct a literature search 

4) Critically appraise the literature 

5) Identify the applicable data 



The questions to be answered by this approach are. (1) Is the study valid? (2) 



What were the results? and (3) Do the results apply to my student(s)? 





The first step is defining the problem. With evidence-based practice, the problem to be solved is 



formulated as a question and the question drives the literature review. The statement of the 
problem is critical to the literature search. Included in the problem are the student description, 
the problem to be examined, the comparative approaches to be used and the outcome of 
interest. A statement might read: “For students with verb choice problems (student description), 
would it be preferable (outcome) to use a phrasal verb as opposed to a single word verb 
(comparative approach)?” 

The literature search is then conducted based on the question mat has been formulated. Each 
search system uses slightly different methodology, and the user is encouraged to become 
familiar with his or her institution’s system. Taking the time initially to lean the system saves the 
user hours of wasted time and the indescribable frustration of receiving too broad or too narrow 
a search. The goal of the literature search is to obtain the mist accurate research data available. 
The literature then is evaluated systematically using a standardized approach. The core of the 
evaluation is to determine the validity of the findings. The study design is used as the initial 
screen to examine validity. Generally, study designs with more control have higher levels of 
validity. The highest level of the evidence is produced by a randomized double blind study, next 
cohort studies, and then control studies. The lowest level of data is provided with case studies. 



(Hayward, Wilson, and Tunis 1995) 




In the evidence-based methodology, studies are divided into those that address specific 



language problems. Integrated studies are presented as a systematic overview, meta-analysis, 
economic evaluation, guideline, or a decision analysis. For each type of study, the founder of 
the evidence-based approach recommend specific question for appraisal of me internal validity 
of the research (see Boxes 2-5). 



Box 2. Critical Appraisal of Course of Action Studies 

Are the findings vaiid? 

Primary Guides 

Was assignment of students to studies randomized? 

Were all of the students who entered the study accounted for? 
Was follow-up complete? 

Secondary Guides 

Were students, instructors, and study personnel “blind” to action? 
Were the groups similar at the start of the study? 

What were the results? 

How large was the effect? 

How precise was the estimate of the effect? 

Will the results help me to help my students? 

Can the results be applied to my students? 

Were all important outcomes considered? 

Are the benefits worth the additional study time and work 
on the part of the student? 



Box 3 Critical Appraisal of Diagnostic Studies 

Are the results of the study valid? 

Primary Guides 

Was there an independent, blind comparison with a reference standard? 
Did the sample include an appropriate spectrum of students? 






Secondary guides 

Were the methods for performing the study described in sufficient detaii to permit repiication? 

What are the results? 

Is data necessary for any type of calculations? 

Will the results help me to help my students? 

Will the reproducibility of the study and its interpretation be suitable for my situation? 

Are the results applicable to my students? 

Will the results change my teaching methods? 

Will students be better off as a result of the study? 



Box 4 Critical Appraisal of Prognostic Studies 

Are the results of the study valid? 

Primary guides 

Was there a well-defined and representative sample of students in the study? 
Was follow-up long and complete? 

Secondary guides 

Were objective and unbiased outcome criteria used? 

Was there adjustment for important prognostic factors? 

What are the results? 

How large is the likelihood of the outcome event(s) in a specified period of time? 
How precise are the estimates of likelihood? 

Will the results help me to help my students? 

Were the students in the study similar to my own? 

Will the results lead directly to selecting a course of action? 

Are the results useful for advising students? 



Box 5 Guidelines for Critical Appraisal of Validity of Articles Integrating Studies 

Overview 

Did the review address a well-focused question? 

Were the criteria used to elect articles for inclusion explicit and appropriate? 

Practice Guidelines 

Were the options and outcomes clearly specified? 

Did the guidelines use and explicit process to identify, select, and combine elements? 

Decision Analysis 

Did the analysis faithfully model an important decision? 

Was valid evidence used to develop probabilities and utilities? 

Economic Analysis 

Were two or more clearly described alternatives compared? 

Were the expected consequences of each alternative based on valid evidence? 



How does the evidence-based practice differ from the classic social science research? 





Evidence-based practice is proposed as a simpie, standardized process for a criticai anaiysis of 



the research iiterature and appiication to ianguage educationai uses. The current focus on 
outcomes research has supported its use. 

There are some differences in the terminoiogy used for evidence-based practice and that used 
with the ciassic sociai science research. Variabies in the evidence-based practice are caiied 
predictors and outcomes. In the quantitative worid these are independent and dependent 
variabies. There are aiso some differences in design terminoiogy. Evidence-based practice 
uses the designs: crossover, experimentai, cohort, case-controi design, and case study. Paraiiei 
research designs in the medicai modei are experimentai, quasi-experimentai, correiationai, and 
descriptive. The sociai science paradigm inciudes quaiitative designs (grounded theory, 
phenomenoiogy, ethnography) that are designed to describe and expiore phenomena in a 
naturaiistic setting. The quaiitative approach is designed for therapy generation, but because it 
most often does not impose controi in the situation, the evidence-based modei considers this 
iower ievei date. 

Another major difference deais with the criticai anaiysis that is used to examine studies. The 
experimentai and quasi-experimentai modei usuaiiy examines the study design’s internai and 
externai vaiidity. Internai vaiidity addresses whether the resuits of the study can be attributed 



to the action of the independent variabie and not other factors. The design is a major factor in 




determining whether internai vaiidity is threatened. Externai vaiidity is the degree to which the 



study findings can be generaiized to the to the target student popuiation. How representative the 
sampie is of the student popuiation of interest in a major factor in externai vaiidity. Threats to 
internai and externai vaiidity are iisted in Box 6. 



Box 6. Threats to Internai and Externai Vaiidity 

Threats to Internal Validity 

History 

Seiection 

Maturation 

Testing 

Instrumentation 

Threats to External Validity 

Hawthorne effect 
Novelty effect 
Experimenter effects 
Measurement effects 



Understanding these more detailed threats allows the teacher to evaluate the validity of 
the study more objectively and use a standardized approach. The reader is referred to 
a language research text for review of this content. Probably the other major way that evidence- 
based practice differs from the classic social science research is the findings are expressed as 
confidence intends rather than p values. Confidence intervals allow the reader to appreciate the 
precision of sample estimates. For example, a 95% confidence interval is an estimate of the 



population parameter 95% of the time. A higher confidence level (99%) is more likely to 





include the population estimate than a lower one (90%). Confidence levels are set prospectively 



by the investigator when the study is designed. 
Conclusions 



Evidence-based practice allows the teacher to provide instruction based on the most recent 
and valid research literature. It is a simple, standardized procedure for critical analysis 
of the literature and application to practice that builds on and enhances classroom expertise. 
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